Computer graphics processing is an intricate process used to create images that depict virtual content for presentation on a display. Modern 3D graphics are often processed using highly capable graphics processing units (GPU) having specialized architectures designed to be efficient at manipulating computer graphics. The GPU is a specialized electronic circuit designed to accelerate the creation of images in a frame buffer intended for output to a display, and GPUs often have a highly parallel processing architecture that makes the GPU more effective than a general-purpose CPU for algorithms where processing of large blocks of data is done in parallel. GPUs are used in a variety of computing systems, such as embedded systems, mobile phones, personal computers, tablet computers, portable game devices, workstations, and game consoles.
Many modern computer graphics processes for video games and other real-time applications utilize a rendering pipeline that includes many different stages to perform operations on input data that determine the final array of pixel values that will be presented on the display. In some implementations of a graphics rendering pipeline, processing may be coordinated between a CPU and a GPU. Input data may be setup and drawing commands may be issued by the central processing unit (CPU) based on the current state of an application (e.g., a video game run by the CPU) through a series of draw calls issued to the GPU through an application interface (API), which may occur many times per graphics frame, and the GPU may implement various stages of the pipeline in response in order to render the images accordingly.
Most stages of the pipeline have well defined inputs and outputs as data flows through the various processing stages, and any particular implementation may include or omit various stages depending on the desired visual effects. Sometimes various fixed function operations within the graphics pipeline are implemented as hardware modules within the GPU, while programmable shaders typically perform the majority of shading computations that determine color, lighting, texture coordinates, and other visual values associated with the objects and pixels in the image, although it is possible to implement various stages of the pipeline in hardware, software, or a combination thereof. Older GPUs used a predominantly fixed function pipeline with computations fixed into individual hardware modules of the GPUs, but the emergence of shaders and an increasingly programmable pipeline have caused more operations to be implemented by software programs, providing developers with more flexibility and greater control over the rendering process.
Generally speaking, early stages in the pipeline include computations that are performed on geometry in virtual space (sometimes referred to herein as “world space”), which may be a representation of a two-dimensional or, far more commonly, a three-dimensional virtual world. The objects in the virtual space are typically represented as a polygon mesh set up as input to the early stages of the pipeline, and whose vertices correspond to the set of primitives in the image, which are typically triangles but may also include points, lines, and other polygonal shapes. Often, the process is coordinated between a general purpose CPU which runs the application content, sets up input data in one or more buffers for the GPU, and issues draw calls to the GPU through an application interface (API) to render the graphics according to the application state and produce the final frame image.
The vertices of each primitive may be defined by a set of parameter values, including position values (e.g., X-Y coordinate and Z-depth values), color values, lighting values, texture coordinates, and the like, and the graphics may be processed in the early stages through manipulation of the parameter values of the vertices on a per-vertex basis. Operations in the early stages may include vertex shading computations to manipulate the parameters of the vertices in virtual space, as well as optionally tessellation to subdivide scene geometries and geometry shading computations to generate new scene geometries beyond those initially set up in the application stage. Some of these operations may be performed by programmable shaders, including vertex shaders which manipulate the parameter values of the vertices of the primitive on a per-vertex basis in order to perform rendering computations in the underlying virtual space geometry.
To generate images of the virtual world suitable for a display, the objects in the scene and their corresponding primitives are converted from virtual space to screen space through various processing tasks associated with rasterization. Intermediate stages include primitive assembly operations that may include various transformation operations to determine the mapping and projection of primitives to a rectangular viewing window (or “viewport”) at a two dimensional plane defining the screen space (where stereoscopic rendering is used, it is possible the geometry may be transformed to two distinct viewports corresponding to left and right eye images for a stereoscopic display). Primitive assembly often includes clipping operations for primitives/objects falling outside of a viewing frustum, and distant scene elements may be clipped during this stage to preserve rendering resources for objects within a range of distances for which detail is more important (e.g., a far clipping plane). Homogeneous coordinates are typically used so that the transformation operations which project the scene geometry onto the screen space plane are easier to compute using matrix calculations. Certain primitives, e.g., back-facing triangles, may also be culled as an optimization to avoiding processing fragments that would result in unnecessary per-pixel computations for primitives that are occluded or otherwise invisible in the final image.
Scan conversion is typically used to sample the primitives assembled to the viewport at discrete pixels in screen space, as well as generate fragments for the primitives that are covered by the samples of the rasterizer. The parameter values used as input values for each fragment are typically determined by interpolating the parameters of the vertices of the sampled primitive that created the fragment to a location of the fragment's corresponding pixel in screen space, which is typically the center of the pixel or a different sample location within the pixel, although other interpolation locations may be used in certain situations.
The pipeline may then pass the fragments and their interpolated input parameter values down the pipeline for further processing. During these later pixel processing stages, per-fragment operations may be performed by invoking a pixel shader (sometimes known as a “fragment shader”) to further manipulate the input interpolated parameter values, e.g., color values, depth values, lighting, texture coordinates, and the like for each of the fragments, on a per-pixel or per-sample basis. Each fragment's coordinates in screen space correspond to the pixel coordinates and/or sample coordinates defined in the rasterization that generated them. In video games and other instances of real-time graphics processing, reducing computational requirements and improving computational efficiency for rendering tasks is a critical objective for achieving improved quality and detail in rendered graphics.
Each stage in conventional graphics rendering pipelines is typically configured to render graphics for traditional display devices, such as television screens and flat panel display monitors. Recently, an interest has arisen for less traditional display devices, such as head mounted displays (HMDs), and less traditional rendering techniques, such as foveated rendering. These non-traditional display technologies present unique opportunities for optimizing efficiency in graphics rendering pipelines.
It is within this context that aspects of the present disclosure arise.